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We report a simple extension of a model for noise reduction in signal processing, already intro- 
duced by Mourik et al, in the presence of binary coupling vectors, which turns to be more useful for 
practical and engineering implementations. We also compute annealed approximation which gives 
an upper bound of the correct critical number of noise-sources. Finally, we also find that the full 
RSB Parisi solution just above the AT line is an universal function for any symmetric distribution 
of the coupling vectors. 



PACS numbers: 64.60.Cn, 89.70.+c,87.10.-he, 02.50.-r 

I. THE MODEL 

In a recent paper Mourik et al. investigated the statistical properties for the problem of noise reduction (NR) 
in signal processing. For this model one considers A^-detectors receiving a signal mixed with noise from p sources. 
The total input on a source from the detector j reads 



where S and are respectively the amplitudes of signal and of the noise from /i-th source, ^^'s are random gaussianly 
distributed variables, and a^'s fixed numbers. The goal is to find a linear combination of the inputs in order to reduce 
the noise and at the same time have a clear signal. By using a A^-dimensional weight vector as a "filter" , the sum of 
observable inputs xj weighted with such a vector, reads 

'^=^E^.-. = ^E-^.-. + EE^- (2) 
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Now, in order to reduce the noise from the local field h^' — Jj^^ /^/N and have the signal part SJ2j JjCLj/VN 
constant we use the strategy which directly minimize the quantity \h^\. If the number of the noise sources p 

is small, it is straightforward to find the optimal solution which minimizes the "cost" \h^\- However, as the number 
of the noise sources increases, in general, it becomes hard to solve such optimization problem. In this case, it is 
natural to wonder how many configurations J = ±1 do exist when the phase space is restricted to < fc, where k 
is the noise tolerance. For each configuration of the noise, the number of the solutions J which satisfy the previous 
condition can be expressed as 

JV{J)^Tr^j^]X^^e{k'-{hn') (3) 

and the "entropy" of the solution space is given by the its logarithm S{J) = \og^f{J). 

In order to investigate the typical properties of the entropy, one usually averages the above quantity with respect 
to the gaussian noise source ^, namely 

< S{J) >^ = < logAA(J) >^ . (4) 
The previous quenched average can be performed by means of the usual replica method |^ , thus giving 

p n 

< AA"( J) >^=< Tr^ n n0(^' - C'a f ) >| ■ (5) 

/J — la— 1 

Therefore, after performing the average with respect to the gaussian randomness and introducing auxiliary variables 
of integrations, the free energy simply reads 
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where 
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As usual, the previous quantities should be evaluated in the limit N oo, by making an ansatz for the matrices 
qaf3 and g^^. In the next section, we consider the replica symmetric solution. 



II. RS SOLUTION 



Within the replica symmetric ansatz 
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it is straightforward to check that the entropy per system size N reads, in the limit n — > 



log < 7V( J) >£ 

^^Siq,q) 
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The stationary points of the entropy with respect to q and q 

dS{q,q) ^ dS{q,q) 
dq dq 
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give the optimal number of the noise sources as a function of the noise tolerance k. Moreover, in order to obtain 
the critical number of the noise source etc, beyond which the solution space shrinks down to zero, we have to estimate 
the expression (Jill) in the limit q^l, viz. 



2(l + fc2)iJ(fc) 



(12) 



We thus get the same result as ||l[ except for a factor 2/7r, just as in the case of the random pattern storage problems 

/2 (large k hmit) and ai\k) 2 (l + 4A:/\/27r) /it (small k limit). 




[^,||. Therefore, one obtains a 

At fixed k, it is important to compare the value of the critical capacity with the AT line, beyond which the 
saddle point solution turns to be unstable against transverse fluctuations. Because of the inversion symmetry of the 
constraints, the saddle point equations dO) are satisfied with the solution g = 0, independent of a and k. The stability 
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of such solution is determined by studying the eigenvalues of the Hessian around the saddle point solution. Therefore, 
the AT line is found to be 



^ [1 - 2H{k)f 

aAT(fc) = 7^——^ — (13) 

as in the continuous case. So, qualitatively we get the same results as ^ where a discontinuous phase transition seems 
to appear for large k. 

In order to give an upper bound for ac{k), we also investigate the optimal number of the noise source using annealed 
approximation So we replace the quenched average by the following annealed average 

< S{J) log < M{J) >| . (14) 
Then we introduce the "magnetization" M ~ Jjl\rN and rewrite the number of the configurations M as 

\t should be noticed that in the quenched calculation, the contribution of the magnetization M. in the entropy becomes 
0(-\/iV) and vanishes in the thermodynamical limit. In the annealed calculation, the average with respect to ^ is a 
simple gaussian integral and we obtain the entropy per system size as 

^anneal = tt log [1 - H{k)\ + log (2cOSh(7W)) . (16) 

The saddle point equation with respect to TW, that is, dS jdM. = leads to A1 = 0, and finally we obtain 

^anneal ^ ^ j^^g _ 2H [k)] + log2. (17) 

As it is well known, due to the concavity of the logarithm, the annealed free energy is a lower bound for the correct 
free energy. From this argument and the fact that the quenched entropy cannot be negative, we obtain a lower bound 
on the zero temperature entropy 5^"*"^^ From (p7|), we get Q;™"°'''(fc), which reads 

---(^)^- log[l-ti/(fc)] -~' 

According to this rigorous condition, RS result ai"' turns out to be larger than a|^""'''''. Therefore, we conclude that 
RS solution is not enough to estimate correctly and more steps of RSB are needed. We plotted a|^""°''' in FIG. 1. 

For large k we get af'''''^^{k) ~ (^-s/^logi^ fce'='/^ and for smaU k af^'''"'\k) ~ - log2/log (^y/2j^k^ . 

In order to estimate the critical number of the noise source, ac, we may use the another criterion, that is, ofg at 
which the zero temperature entropy of the RS saddle point becomes negative. By numerical calculations, we checked 
that the entropy S of the RS saddle point becomes negative a,t q = q = for all k. This leads to 

as = (19) 

and this RS solution is locally stable. Therefore, we conclude that as gives one of the good approximations for the 
critical number of the noise source. 



III. PARISI SOLUTION 

As in |l] the finite step RSB solution is found to be unstable against traverse fluctuations, signaling the necessity 
of a full RSB solution. Analytical results can be obtained only close to the AT line, where the order parameters are 
small {qaf3 {a — aAT)/o.AT), and then the free energy, or equivalently the saddle point equations, can be expanded 
in terms of qap and qap ■ The expressions involved in here are quite similar to the SK model, where for small value of 
the order parameter, the free energy has, besides a quadratic term, a cubic term Tiq'^ and a quartic term X^Q/s'Za/Jj 
whose presence allows a full-fledged replica symmetry breaking solution . It is straightforward to see that the saddle 
point equations, obtained by varying ^ with respect to qa/3 and (jajs, coincide with the equation for the continuum 
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J case, thus giving the same Parisi solution, i.e. Hnear up to a breakpoint and then constant^ = qsA, for any value 
of k as long as k'^qsA 1. 

Finally, it is worth noticing that the saddle point equation with respect to (jap, involving the distribution of the 
couplings J, gives, in the proximity of the AT line, qap ^ Qap for any kind of symmetric distribution of the coupling 
vectors J . Therefore we conclude that the Parisi solution just above the AT line is an universal function, whereas 
sizable differences may arise far away from the AT line. 
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FIG. 1. RS solution, annealed approximation (AA), AT line and RS solution by Mourik et al. 
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FIG. 1 
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